Skip to content

[SPARK-11769] [ML] Add save, load to all basic Transformers#9755

Closed
jkbradley wants to merge 4 commits intoapache:masterfrom
jkbradley:transformer-io
Closed

[SPARK-11769] [ML] Add save, load to all basic Transformers#9755
jkbradley wants to merge 4 commits intoapache:masterfrom
jkbradley:transformer-io

Conversation

@jkbradley
Copy link
Member

@jkbradley jkbradley commented Nov 17, 2015

This excludes Estimators and ones which include Vector and other non-basic types for Params or data. This adds:

  • Bucketizer
  • DCT
  • HashingTF
  • Interaction
  • NGram
  • Normalizer
  • OneHotEncoder
  • PolynomialExpansion
  • QuantileDiscretizer
  • RFormula
  • SQLTransformer
  • StopWordsRemover
  • StringIndexer
  • Tokenizer
  • VectorAssembler
  • VectorSlicer

CC: @mengxr

@SparkQA
Copy link

SparkQA commented Nov 17, 2015

Test build #46047 has finished for PR 9755 at commit 13ba079.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * class HashingTF(override val uid: String)\n * class Normalizer(override val uid: String)\n * class SQLTransformer (override val uid: String) extends Transformer with Writable\n * class Tokenizer(override val uid: String)\n

@SparkQA
Copy link

SparkQA commented Nov 17, 2015

Test build #46053 has finished for PR 9755 at commit 7e0128c.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * class HashingTF(override val uid: String)\n * class Interaction @Since(\"1.6.0\") (override val uid: String) extends Transformer\n * class Normalizer(override val uid: String)\n * class SQLTransformer @Since(\"1.6.0\") (override val uid: String) extends Transformer with Writable\n * class Tokenizer(override val uid: String)\n

@SparkQA
Copy link

SparkQA commented Nov 17, 2015

Test build #46054 has finished for PR 9755 at commit d56fab9.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * class HashingTF(override val uid: String)\n * class Interaction @Since(\"1.6.0\") (override val uid: String) extends Transformer\n * class Normalizer(override val uid: String)\n * class SQLTransformer @Since(\"1.6.0\") (override val uid: String) extends Transformer with Writable\n * class Tokenizer(override val uid: String)\n

@jkbradley
Copy link
Member Author

jkbradley commented Nov 17, 2015

Don't commit yet. I just realized I forgot to overload the load method, so it's not Java compatible.

@jkbradley
Copy link
Member Author

jkbradley commented Nov 17, 2015

OK hopefully good pending tests

@SparkQA
Copy link

SparkQA commented Nov 17, 2015

Test build #46100 has finished for PR 9755 at commit 31b6bc4.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * class HashingTF(override val uid: String)\n * class Interaction @Since(\"1.6.0\") (override val uid: String) extends Transformer\n * class Normalizer(override val uid: String)\n * class SQLTransformer @Since(\"1.6.0\") (override val uid: String) extends Transformer with Writable\n * class Tokenizer(override val uid: String)\n

asfgit pushed a commit that referenced this pull request Nov 17, 2015
@jkbradley @mengxr
[SPARK-11769][ML] Add save, load to all basic Transformers
e7f9016 
This excludes Estimators and ones which include Vector and other non-basic types for Params or data.  This adds:
* Bucketizer
* DCT
* HashingTF
* Interaction
* NGram
* Normalizer
* OneHotEncoder
* PolynomialExpansion
* QuantileDiscretizer
* RFormula
* SQLTransformer
* StopWordsRemover
* StringIndexer
* Tokenizer
* VectorAssembler
* VectorSlicer

CC: mengxr

Author: Joseph K. Bradley <joseph@databricks.com>

Closes #9755 from jkbradley/transformer-io.

(cherry picked from commit d98d1cb)
Signed-off-by: Xiangrui Meng <meng@databricks.com>
@mengxr
Copy link
Contributor

mengxr commented Nov 17, 2015

LGTM. Merged into master and branch-1.6. Thanks!

@asfgit asfgit closed this in d98d1cb Nov 17, 2015
@jkbradley jkbradley deleted the transformer-io branch November 17, 2015 20:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@jkbradley @SparkQA @mengxr